Evolutionary Dynamics and the Phase Structure of the Minority Game 
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We show that a simple evolutionary scheme, when applied to the minority game (MG), changes 
the phase structure of the game. In this scheme each agent evolves individually whenever his wealth 
reaches the specified bankruptcy level, in contrast to the evolutionary schemes used in the previous 
works. We show that evolution greatly suppresses herding behavior, and it leads to better overall 
performance of the agents. Similar to the standard non-evolutionary MG, the dependence of the 
standard deviation a on the number of agents N and the memory length m can be characterized by 
a universal curve. We suggest a Crowd- Anticrowd theory for understanding the effect of evolution 
in the MG. 
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PACS numbers: 89.65. Gh, 87.23. Ge, 02.50.Le 



I. INTRODUCTION 

Complex adaptive systems consist of agents using 
adaptive strategies to compete for limited resources. As 
changes in the global environment are induced by the 
agents themselves, it is important to study dynamics 
of such systems. The minority game (MG), proposed 
by Challet and Zhang0, is a prototypical agent-based 
model that can be analyzed using the tools of statisti- 
cal mechanics. The game captures some essential fea- 
tures of complex adaptive systems in which agents with 
limited information and rationality compete for limited 
resources. A key question in the study of agent-based 
models is, how evolution changes the behaviors of the 
agents. 

There have been a few studies on the effect of evo- 
lution in the minority game. In the context of a sim- 
ple evolutionary minority game, Johnson et al. found 
that the agents universally self-segregate into two oppos- 
ing extreme groups Hod and Nakar, on the other 
hand, claimed that clustering of cautious agents emerges 
in a "tough environment" where the penalty for losing is 
greater than the reward for winning . Chen et al. EJ 
derived a general formalism to understand the dynamical 
mechanism for the transition from segregation to cluster- 
ing. They found that the effective rate of evolution plays 
an important role in determining the resulting steady- 
state population distribution. These studies have focused 
mainly on population distribution. Li et al. .73 j on the 
other hand, studied how evolution can help to improve 
the overall performance of the agents in the original MG. 
Starting from the adaptive MG proposed by Challet and 
Zhang 0] , Li et al. introduced an evolutionary scheme in 
which all poorly performing agents evolve synchronously 
at every r = 10, 000 steps. Agents are ranked by their 
gains, and those ranked at the bottom p percent (p=10%, 
20%, 30%, 40%, etc) are forced to change their strate- 
gies at these pre-specified steps. In order to make the 
evolution process smooth, not all the agents ranked at 
the bottom will change their strategies, but only 50% of 



those (chosen randomly) have to do so. Those who are 
chosen to evolve replace the current strategies with new 
randomly picked ones. They reported that with evolu- 
tion the performance is significantly better; but the phase 
structure, characterized by the so called Savit curve 0], 
remains similar to that of the original non-evolutionary 
MG. A later study based on a variant of the evolution- 
ary scheme used in Ref. led to a similar conclusion, 
but with better overall performance of the agents. 

When dealing with models of heterogeneous agent pop- 
ulation, it makes sense to use an evolutionary scheme 
in which agents evolve individually, instead of syn- 
chronously at specified times. In this paper we adopt the 
simple scheme used in the EMG 0], in which an agent 
becomes bankrupted and is replaced whenever its accu- 
mulated wealth is below a given threshold. With this 
simple scheme we found that herding behavior has dis- 
appeared when the memory length (m) is small, and the 
Savit curve obtained is significantly different from that 
of the original MG. 



II. NUMERICAL RESULTS 

Let us first briefly describe the minority game model. 
The game concerns a population of N (odd number) het- 
erogeneous agents with limited capabilities, who repeat- 
edly compete to be in the minority group. After each 
round the winners gain a point and the losers lose a point. 
Each agent holds S strategies. Each strategy is a look- 
up table listing the strategy's prediction of the minority 
group given the record of the most recent m minority 
groups. There are total 2 2 number of possible strate- 
gies, so the larger the value of m, the greater the process- 
ing power of the agents. Virtual points are accumulated 
for each of the strategies the agent has, and he uses the 
most successful strategy available to him. To include the 
effect of evolution we assign wealth w to each agent; w 
will increase/decrease by one when the agent wins/loses. 
The agent will be replaced if his wealth is below a thresh- 
old — d (d > 0); the new agent chooses his S strategies 
randomly and his wealth is initialized to zero. The dis- 
tribution of strategies gradually evolves as the game goes 
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FIG. 1: a 2 /N vs 2 m /N for the MG with and without evo- 
lution, d — 256 is used for the evolutionary MG. The results 
are obtained by averaging over eight independent runs 



We have done extensive simulations with ./V = 
51,101,201,401; m = 1,2, •••,10; S = 1,2; and d = 
4, 16, 64, 256, 1024, 4096. The number of time steps used 
is set to be T = 800000 x 2 m / 2 + lOOOOd; we have checked 
that this choice of the time step is sufficient for obtain- 
ing steady-state properties of the model. In our simula- 
tions we monitor er, which is the standard deviation of 
the number of agents belonging to one of the groups. The 
smaller the value of a, the larger a typical minority group 
and the better the overall performance of the agents will 
be. The smallest value of a is a = 0.5, which means that 
the difference between the numbers in the minority and 
majority groups is one. The overall performance of the 
agents is at the optimal when a = 0.5. At the other ex- 
treme, when the agents make their choices randomly as 
in the random choice game (RCG), we have a 2 /N = 0.25. 
In almost all our simulations, particularly for small to, 
evolution reduces a significantly. This is illustrated in 
Fig. 1, which shows a 2 /N vs 2 m /N for the MG with and 
without evolution. The results are based on averages over 
eight independent runs, which are enough for obtaining 
accurate averages for the evolutionary MG, as the differ- 
ences among different runs are quite small. For small m 
evolution leads to a dramatic reduction in a as herding 
behavior of the agents is greatly suppressed. The results 
for different N fall to a universal curve. For large m 
the game still approaches the limit corresponding to the 



random choice game and the effect of evolution is small. 

We have also studied the model with the award-to- 
fine ratio (as defined in Ref. |U) R ^ 1. In this case 
the winners get R points while the losers lose a point. 
We found that, for each N, there is an optimal value of 
R = R C (N) > 1 that gives rise to the smallest value of u. 
For R > R C (N) the average wealth is ever increasing and 
there is no steady state. Fig. 2 shows a 2 /N vs 2 m /N for 
R > 1 as compared to the case R = 1. It is clear from 
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FIG. 2: a IN vs 2 m /iV for the evolutionary MG. d = 64 
is used. The results are obtained by averaging over eight 
independent runs 



the figure that a can be further reduced when R > 1. 
We also found that, for to = 1 or 2, the optimal value of 
a = 0.5 can in fact be achieved. 



III. A CROWD-ANTICROWD THEORY FOR 
THE EVOLUTIONARY MG 

We now consider a Crowd-Anticrowd theory 01 to 
understand the effect of evolution in the minority game. 
Our discussion below follows Ref. ^3 • F or simplicity we 
only consider S = 1 and use a reduced strategy space 
(RSS). Numerically the differences between the cases 
with S = 1 and S = 2 are small. RSS is a subset of 
strategies, which span the full strategy space (FSS). Con- 
sider, for example, an RSS for to = 2, consisting of the 
following eight strategies: 



U = {-1-1-1-1}, {+1 + 1-1-1}, {+1-1 + 1-1}, {-1 + 1 + 1-1} 
U = {+1 + 1 + 1 + 1}, {-1-1 + 1 + 1}, {-1 + 1-1 + 1}, {+1-1-1 + 1} 

I 



(1) 

(2) 



Here ±1 indicate the prediction of a strategy given one U} are either uncorrelated (with the Hamming distance 
of the four possible histories. Any two strategies in {U, 
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FIG. 3: Histogram for the number of appearances of all possi- 
ble histories. N = 101, S — 2, m = 6, and d = 256. For com- 
parison the corresponding histogram for the non-evolutionary 
MG is also plotted 



2 m /2) or anti-correlated (with the largest Hamming dis- 
tance 2 m ). For a given m there are total 2P (P = 2 m ) 
strategies with P pairs of anti-correlated strategies (other 
pairs are uncorrelated) in RSS. For each strategy G in 
RSS there is a corresponding anti-correlated strategy G. 
It is believed that the essential features of the game are 
kept when RSS is used instead of the FSS @|. 

Let us evaluate a 2 =< (n+(i) - N/2) 2 > t =< (n + (t) - 
ri-(t)) 2 > /4, where n + and n_ are the numbers of agents 
making the choices +1 and —1 respectively. The average 
is over time step t. In RSS 



n+{t) — n_ (t) = 



2P 

E 

G=l 



li(t) 
a G n G, 



where fi(t) denotes the current history, a G = ±1 is the 
response of strategy G to the history bit-string /x(t), and 
n G is the number of agents using strategy G at time t. 
We have 



a 2 = 



2P 



,**(*) 



< a G x 'n G a G ) 'n G , > t 



(3) 



G=1,G'=1 



In our simulation we found that the system visits all pos- 
sible histories equally. This is illustrated in Fig. 3, which 
shows a histogram of the number of visits for each m-bit 
history. In order to compare with the non-evolutionary 
MG, we use S — 2 in the figure; the result for S = 1 is 
essentially the same. As we can see from the figure, the 
system does not visit all possible histories equally in the 
non-evolutionary MG. 

For the evolutionary MG we can thus replace an av- 
erage over time by an average over all possible histo- 
ries. Now the double sum can be broken down into 
three parts, based on the correlation between the strate- 
gies. Given < a G a G >t= 1, < clgo-g >t= — 1, and 



< a G a G > > t «< a G a G > >h= (for G' ^ G and G' ^ G), 
where < • • • >h indicates average over all possible histo- 
ries, we can write the double sum as 



2P 



2P 



(4) 



<U G>t < n G n G >t 

G=l G=l 

I p 

= ^ 22 [< n% > t -2 < n G n G > t + <n 2 G > t ] (5) 

G=l 

! p 

= 4 E < ( n G - n G ) 2 > t (6) 

G=l 

In the above derivation we have tacitly assumed that 
{n G } change very slowly so the averages over {a G } can be 
done while holding {n G } constant. This is in the spirit of 
adiabatic approximation; it is valid because the evolution 
rate is low (as long as d is not too small). 

Let np be the number of pairs of agents holding anti- 
correlated strategies and ns be the number of agents 
holding unpaired strategies. Only the agents holding un- 
paired strategies contribute to a; the game behaves es- 
sentially as an RCG with ns number of agents. Thus we 
have a 2 = 0.25ns, or 



a 2 jN = 0.25s, 



(7) 



where s = ns/N. Note that 2np + ns = N , np can be 
written as np — N(l — s)/2. To determine s we need to 
consider the evolutionary dynamics of the game. At the 
steady state we have the following balance equation 



2rpnp(l — p(m, N)) — rsnsp(m, N), 



(8) 



where rp and rs are the bankruptcy rate for the paired 
agents (pair breaking rate) and the bankruptcy rate for 
the unpaired agents respectively; p(m, N) is the proba- 
bility that a new pair is formed when a bankrupted agent 
is replaced. One can estimate p(m, N) as 

p(m,N) = l-(l-l/2P) ns w l-exp(-0.57Vs/2 m ) (9) 

It is somewhat difficult to estimate rp and rs- It can be 
argued that rp/rs does not sensitively depend on m and 
N . The equation for s is then 



rp_ 

rs 



(1 



P(s/z) 
1 -p(s/z) : 



(10) 



where z = 2 m /N. The solution s is a function of z. Thus, 
when we plot a 2 /N vs z — 2 m /N, the curves fall to a 
universal curve. Since p(x) is a monotonically decreasing 
function of x, the universal curve s(z) obtained from the 
above equation will be a monotonically increasing func- 
tion of z. In the limit z — > oo we have p(s/z) — > 0; this 
leads to s -» 1 or a 2 /N = 0.25. Thus z ->• oo is the RCG 
limit. For R > 1, rp/rs decreases; this leads to smaller 
values for s and a. All these are in agreement with the 
simulation results. Thus the Crowd- Anticrowd picture 
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provides a qualitative understanding of the evolutionary 
MG. 

In conclusion, we have shown that, when a simple evo- 
lutionary scheme is applied to a heterogeneous popula- 
tion of agents, herding behavior in the MG is greatly 
suppressed. The dependence of the standard deviation 
a on the number of agents N and the memory length m 



can be characterized by a universal curve. In addition, 
we demonstrated that a Crowd- Anticrowd theory can be 
used to understand qualitatively the effect of evolution 
in the MG. 
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